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I was asked to speak about how knowledge of visual perception could be applied to achieve tele- 
operation and virtual worlds. After a little thought I decided that reversing subject and object makes 
the question more interesting and provocative. The practice of teleoperation and virtual world tech- 
nology challenges our understanding of perception, and in the following I try to explain what I mean. 
But first let me say that vision science incorporates a vast amount of useful knowledge about the 
capabilities of the visual system. Fortunately, most of it is well-codified in the handbooks, of which 
the most recent and complete is Boff, Kaufman, and Thomas’s Handbook of Visual Perception and 
Performance (1989). Unfortunately, this codified knowledge does not prepare us for the surprises 
attendant on the sensory and motor transformations effected by the new technologies. Instead, the 
surprises refer us to much less well understood areas. Hence the challenge to perceptual science from 
technology. 


World Regularities and the Habitual 

New technologies make it easy to transform spatially, temporally, and dynamically the relations 
among sensory inputs and motor outputs. For example, nothing could be simpler than making your 
hand appear to reach to the left when you intended it to reach to the right, to make you step up when 
you should step down, make the usually stable visible world appear to move as you move, and so on. 
The consequences can be errors in performance, disorientation, and a form of malaise like motion 
sickness. These examples make us realize to what extent our perceptions are dependent upon the 
regularities and habitual circumstances of our normal world and the dynamics of our actions in it. 

We are preadapted to these ordered states in multiple ways that are revealed by the effects of apply- 
ing the new technologies to the human-machine interface. Compared to our knowledge of basic 
visual processes, our understanding of these orderings is limited but not insignificant. Some of it has 
been relegated to the fringes of the field as aberrant and bizarre forms of perception. The following 
table enumerates several types of transformation and examples. 


Type 

Spatial Incongruity 
Temporal Asynchrony 
Altered Dynamics 
Deprivation 


Examples 

Japanese Illusion 
Sensory Rearrangement 

Phase Shift 
Time Delay 

Microgravity 
Force Amplification 

Blank Visual Field 
High Noise Level 
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Some of these non-habitual conditions may cause initial errors and disorientations. Some show 
adaptation during continued exposure in the form of error reduction, better orientation, and reduction 
of motion sickness. Adaptation requires maintenance of a transformed set of regularities. If the trans- 
forms are complex and/or if the coupling is noisy, adaptation may be either very slow or not possible 
at all. However useful adaptation may be, it does take time and represents an additional load on the 
processing capacity of the system, sometimes called a cognitive load. 

Telepresence and Presence 

A particularly intriguing problem, related to the above-mentioned considerations, is the creation 
of telepresence. It raises the question of what conditions specify the sense of presence — a question 
that seems at first blush so obvious as to require no thought. It is the first axiom of perceiving, 
namely, that the observer be there to perceive. It implies a here and now spatial juxtaposition 
(confrontation) of object and observer. We can begin to identify conditions for the sense of presence 
as: 

1. Independent perception of an observer and an object external to the observer. 

2. Covariation of sensed observer movement independent of the sensed object yields the sense 
of an object external to the observer. 

3. Observer has access to the sensed object at will. 

Achievement of these conditions is, in turn, related to the regularities and habitual circumstances 
discussed above. 
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